1. Field of the Invention
The present invention relates to a human-computer interface using neuroelectric signals recorded from the user's scalp, i.e., electroencephalograms (EEGs).
2. Description of Related Art
Conventional computer user interfaces are driven solely by behavioral responses (i.e. muscle activity) by the user with a keyboard, mouse, joystick, touchscreen, pen or similar device, or a data glove in a virtual reality system. In this type of system, the user receives information from the computer screen and audio system in the form of visual and auditory stimuli, (or haptic stimuli in a virtual reality system), processes them, and makes a deliberate behavioral action which the computer interprets and acts upon. Computers that interpret human speech, or that use devices to determine where the user is looking, are other examples of interfaces that are controlled by behavioral responses from the user. Another type of user interface, which is still in the experimental stage, accepts a user's "thought commands," as measured by EEG signals, to control the movement of a cursor on a display screen.
All these interfaces are intended to allow the user to operate and control the computer system. The computer system has no information about the amounts and types of the user's mental capacities currently being utilizied, or even about the user's state of alertness. This results in a situation in which the overall efficiency of the human-computer system is less than it might be. For example if the user is mentally overloaded, or at the other extreme, if the user is drowsy, the overall performance of the human-computer system will be limited by the ability of the user to process and respond to the information presented by the computer.
Advances in technology have resulted in more complex computer based or computer controlled systems which can overwhelm the user's ability to process and respond to the information presented. Examples of this include jet fighter planes, air traffic control systems, powerplant and factory control systems, emergency management systems, multi-window displays of complex relations in a large data base, securities trading systems, and video games which increase task difficulty beyond a user's ability. New multimedia and virtual reality technologies are likewise expected to produce situations in which a user is mentally overloaded. At the other end of the mental effort continuum, highly automated computer controlled systems can require so little input from the user that the user can become inattentive or drowsy, for example piloting a commercial airliner. Other situations which can cause boredom and resultant inattention or drowsiness include long duration instrument monitoring tasks such as watching radar or sonar displays for unusual activity.
The lack of on-line knowledge of whether the user is mentally underloaded or overloaded is also a major limitation in computer-aided instruction systems. The use of computer-aided instruction is greatly increasing because of its ability to present material at a pace directed by the user, as compared with traditional instruction in which everyone in the class receives the same material at the same rate. However, unlike a human teacher, a computer-aided instruction system has no way of knowing about the user's mental state and therefore can not optimally adapt the material to her or his needs. For example, when a user answers a question incorrectly during a computerized training program, the system does not know whether the user was not paying attention or whether she or he was trying hard and simply did not know or understand the material. In the former case, an alerting signal could be presented and then the same material could be repeated. In the latter case, it would be useful to know whether or not the user was employing an appropriate strategy to solve the problem. If so, a more detailed explanation of the material which was not understood could be presented. If the user was using the wrong strategy for the problem, an explanation of how to go about solving the problem could be presented. For example, it might be determined that at the time the user made an error when answering a question requiring visualization of how the parts of an engine fit together, he was using 75% of his cognitive capacity; visuospatial systems were at 45% of capacity, while verbal encoding and output systems were at 85%. From this information, the system could conclude that the user was trying to solve the problem with a verbal strategy which was not efficient for the problem at hand and could present the user with information showing him or her how to use a visuospatial strategy to solve the problem. There currently is no way to obtain this information except indirectly by querying the user about his or her mental state. Besides distracting from the flow of the instruction session, this approach can be inaccurate since people are not always aware of their mental state.